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W e examine the use of formative assessment as a tool to assist teachers of 
mathematics to become more mindful developers of curricula. We 
focus on instructional design that is based on careful examination of student 
answers to questions. Empirical studies have shown the effectiveness of form- 
ative assessment for students, and recent theoretical work indicates that the 
positive feedback aspect of formative assessment stimulates self-regulation 
and transformation, processes that are regarded as critical to developing 
intelligence. We apply these ideas of formative assessment to teachers of 
senior mathematics as they rethink instructional design. 



Formative assessment 

When teachers understand what students know and how they think, and then 
use that knowledge to make more effective instructional decisions, significant 
increases in student learning occur. Black and Wiliam (1998) examined 
approximately 250 studies and found that gains in student learning resulted 
from a variety of methods all of which had a common feature: formative 
assessment. This is a form of assessment that uses the data acquired to adapt 
instruction to better meet student need. They concluded that (1) improving 
formative assessment resulted in noticeable increases in student learning; (2) 
there is room for improvement; and (3) there are ways to improve the effec- 
tiveness of formative assessment. 

Formative assessment and feedback 

The distinction between formative and stimulative assessment seems to have 
been made explicit first by Scriven (1967). He favoured stimulative assess- 
ment, but was aware of the preference for formative assessment of his 
colleague Cronbach (1957) who saw it as “part of the process of curriculum 
development” (Scriven, 1967, p. 41). Roos and Hamilton (2004, 2005) 
emphasise the intellectual debt of formative assessment to cybernetic theory, 
via the critical notion of positive feedback. They also link formative assess- 
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ment to a mindful approach to teaching, curriculum development, and 
instructional design. Black and Wiliam (2003, p. 631) also emphasise this 
aspect of formative assessment, when they write: “more will be gained from 
formative feedback where a test calls for the mindfulness that it helps to 
develop.” 

The critical intellectual basis of formative assessment, according to Roos 
and Hamilton, is that positive feedback provides a stimulus to the activities of 
self-regulation and transformation — essential elements, in Piagetian episte- 
mology, for generating higher aspects of intelligence (Piaget, 1963). 

Formative assessment is a way of assisting students to develop, and to help 
them become more mindful and aware, through the processes of self-regula- 
tion and transformation. This is the strength of formative assessment, as 
evidenced empirically by the seminal Black and Wiliam (1998) meta-analysis, 
and theoretically by being based on positive feedback to provide a stimulus to 
these critical thought processes for students (Roos & Hamilton, 2004, 2005). 
Teachers can ask questions of students that provide data for new and 
improved forms of instruction, which was the basis of Cronbach’s interest in 
formative assessment in the 1950s. What teachers can learn from student 
answers to questions may impact directly on the cohort of students being 
taught at the time, and it may also have effects on the next cohort to be taught 
the same content. While the latter outcome is not a common understanding 
of formative assessment, we argue that it is entirely consistent with formative 
assessment seen, in Cronbach’s terms, as part of the process of curriculum 
development (Cronbach, 1957). We have argued, too, that the essential 
features of formative assessment are not simply real-time adjustments to 
curriculum as it is being delivered but are more fundamentally focused on 
feedback to provide mindful revision of curriculum. 

Formative assessment and questioning 

Black and Wiliam (1998) found that not all classroom discussion and ques- 
tioning results in improved learning: discussions and questions in which a 
teacher is looking for a particular response actually inhibits future learning by 
shutting down thoughtful unanticipated, but productive, attempts by students 
to come up with their own answers. A common problem of the use of ques- 
tioning as a means of checking on learning is that it is often unproductive. 
Teachers frequently answer their own questions in less than a few seconds 
without providing students with sufficient time to think about a response, and 
regularly call on the same few students to answer questions (Rowe, 1972, 
1987; Tobin, 1987). 

Interpreting answers as data 

Royall (1997) has written eloquently about the dilemmas facing practitioners 
in interpreting data. He formulates three questions that have wide applica- 
bility to the interpretation of all data. These questions are often confounded, 
and Royall shows clearly how separating them can lead to significant insights 
into the interpretation of data. As practitioners, mathematics teachers gain 
data from their own students — in the form of test answers, classroom 
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answers, and classroom conversations. Royall’s analysis suggests to us that it 
would be very useful for mathematics teachers to ask themselves the following 
three questions when trying to asses student responses to questions: 

• What do I believe now that I have this answer? 

• What do I do now that I have this answer? 

• What is this answer evidence for? 

We argue that a powerful way of connecting formative assessment, with its 
demonstrable benefits in terms of student learning, with teacher questioning 
is to link the two via the approach a teacher takes to student answers. Utilising 
Royall’s data analysis questions in the context of formative assessment leads us 
to formulate to the following general feedback model: 

1 . A teacher asks questions of students, in class or in tests. 

2. Students answer questions, verbally or in writing. 

3. The teacher analyses student answers using one or more techniques of 
data analysis. 

4. The teacher reflects on changing beliefs, need for action, or evidence 
for or against an existing assumption. 

5. The teacher modifies and redesigns curriculum as a response to the 
analysis of student answers. 


Fragmentation of student knowledge 

Preliminary observations 

A common observation in our own teaching of mathematics is that student 
knowledge is commonly fragmented and not connected, and that this gener- 
ally indicates a superficial degree of concept learning (McGowen & Davis, 
2005). An analysis of final examination data of 148 students enrolled in an 
Introductory Algebra class in 2003 at two-year college (College A) in the mid- 
west of the United States documented noticeable differences in the 
percentage of correct students’ responses to pairs of related questions dealing 
with slopes of linear equations and linear inequalities. The question: 

Determine the slope/intercept form of the equation 2x + 3y = 6 

was answered correctly by 87% of students, yet only 49% correctly identified 
the graph of a linear equation with undefined slope. Only 41% of the 148 
students correctly answered both questions. On two related questions dealing 
with linear inequalities, 89% of the students correctly graphed the inequality: 
-3 < x < 5 but only 48% could correctly locate the point (x, y), if x > 0 and 
y < 0. Only 41% correctly answered both questions. 

These observations are consistent with fragmentation of student knowl- 
edge and lack of flexibility in mathematical thinking that both authors have 
seen in similar contexts (Davis & McGowen, 2002; McGowen & Davis, 2002; 
McGowen & Davis, 2005). Flexibility of mathematical thinking also seems to 
be an issue that is paid little attention by a majority of mathematics teachers 
(Warner, Davis & Coppolo, 2002; Warner, Davis, Alcock & Coppolo, 2002). 
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Further evidence of fragmentation of student knowledge 
This phenomenon was further investigated at a large mid-western university 
in the United States (University B) in 2004, where 140 students enrolled in an 
introductory algebra course were asked the following three related questions 
on linear equations and slope on a survey at the beginning and at the end of 
the semester. Student responses to the three questions were examined singly 
and in combination. 


1. Find the equation of the line with slope -7 and a vertical intercept at 3. 

2. Which of the following equations has the given graph? Circle and justify 
your choice. 

a. 3x+2y=6 b. 2x+3y=6 c.2x-3y=6 d. 3x-2'y=6 e. none 



3. Which of these graphs could be the graph of y = 40 - 0.7,x? Circle and justify 
your choice. 





Student responses, summarised in Table 1, indicate that the students 
entering the developmental algebra course had or recalled little prior knowl- 
edge about linear equations and slope. 

By the end of the semester, though there had been some improvement in 
the number of students who correctly answered each of the questions, the 
overall results, summarised in Table 2, are disheartening. 

Less than 20% of the 140 students were able to write a correct linear equa- 
tion given the slope and the y-mtercept by the end of the semester. Further 
examination of the data tells us that of the 140 students, 46 made no attempt 
to answer Question 1. When students’ responses to the first two questions are 
examined jointly we find that only 4 students (3%) got both questions 1 and 
2 correct. Only one of the 140 students answered all three questions correctly. 

This simple analysis raises the obvious question of why so few students 
could answer both of these related questions. It is evidence that students do 
not see questions 1 and 2 as being intimately connected. Depending on an 
instructors’ thinking, it might also lead one to believe that students are 
unable to apply what they do know when confronted with a different context 
— an indication that what little they have learned and remembered is frag- 
mented and unconnected and leads one to believe that their knowledge of 
slope and linear equations is quite fragile. The data indicates that an instruc- 
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Table 1. Pre-test correct responses to questions on linear equations (n=140) 


QUESTION 

# CORRECT 

% CORRECT 

1. Given slope and ^intercept, write equation of 
the line. 

0 

0% 

2. Given graph, select the matching linear 
equation. 

3 

2% 

3. Given the linear equation, select the matching 
graph. 

12 

9 % 


Table 2. Post-test correct responses to questions on linear equations (n=140) 


QUESTION 

# CORRECT 

% CORRECT 

1. Given slope and ^intercept, write equation of 
the line. 

24 

17% 

2. Given graph, select the matching linear 
equation. 

11 

8% 

3. Given the linear equation, select the matching 
graph. 

54 

39% 


tor teaching a course in which related questions are pertinent, needs to make 
a concerted effort to draw out the explicit connections between the ques- 
tions. 

It might be argued that a possible explanation for the very small number 
of correct responses to Question 2 is that the question is misleading and that 
a different choice of graph and equation would have yielded better results. 
Attributing student difficulties to misleading choices in Question 2 fails to 
consider responses to related questions that, when examined together reveal 
other possible underlying sources of error. On Question 1, which asked 
students to write the equation of a line given the slope and ^intercept, 34 
students drew a graph and plotted the slope as the x-intercept or, in a few 
instances, as the ^intercept. Many of these same students labelled the x- and 
y-coefflcients as intercepts on the graph of Question 2. Only 41 students 
attempted to write an equation and 1 7 of them were not successful. 

Students’ written comments further support the conclusion that many of 
them interpreted the slope as an ordered pair and plotted it as an intercept 
or equated the x- and y-coefflcients in the equation with the intercepts on the 
graph. The following were the most common responses: 

“Positive 3 at x-axis (independent); negative 2 aty-axis (dependent).” 
“Positive 3x is visible and -2y.” 

“The x-intercept is 3x; the y-intercept is -2y. ” 

Student tests questions and formative assessment 

The fact that differences in correct answers between deliberately included 
related questions were found on the survey administered at University B, and 
similar differences were found on items typically found on course exams 
which could also be grouped at College A, suggests that this type of analysis is 
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both a formative assessment tool when used to modify current instruction and 
help raise students’ awareness of their own areas of strengths and weaknesses, 
and as a basis for informed curricular decisions and revision. Correctly 
answering a majority or all of a grouping of related questions on a given topic 
is an indicator of the ease or difficulty a student has in adapting to changing 
problem conditions and in reorganising one’s work. A possible step to 
improving student success could be to incorporate formative assessment prac- 
tices that provide teachers with data on what students know and how they 
understand what they are learning. Teachers could then make more 
informed, principled decisions about courses of action that result in 
improved student learning and higher rates of success. 


A pilot formative assessment project 

These experiences led to the design of a pilot formative assessment project, 
September-December 2006, in which three teachers at College A agreed to 
investigate the extent to which introductory algebra students demonstrate 
understanding of, and ability to apply, both concepts and skills in different 
contexts in the topic areas of: (a) linear equations and slope; (b) linear 
systems, and (c) linear inequalities. Each assessment consisted of related 
multiple-choice questions administered prior to introduction of the topic 
during the Fall 2006 semester. After completion of each formative assessment 
and analysis of the data, the researcher and the three instructors were to meet 
to (a) discuss the results, (b) develop possible follow-up instructional tasks 
based on information provided by the assessments, and (c) review the next 
proposed formative assessment prior to its administration. Related questions 
similar to the formative assessment items were to be included on the depart- 
mental introductory algebra final exam. 

Sample results and analysis from the pilot formative assessment project 
(A) Student answers 

Analysis of related questions on the topics of linear equations/slope, linear 
systems, and linear inequalities provided evidence of students’ fragmented 
learning and failure to utilise knowledge and skills learned in one context in 
a different situation. On the first pre-test, students did not distinguish inter- 
cepts from x- and y-coefficients. Several students insisted that an intercept is a 
number value, not a point/ ordered pair. Lack of a principled understanding 
of the concept of slope was also documented. Some students wrote the slope 
as an ordered pair and did not view slope as a ratio. Others used the value of 
the slope as the x-intercept value. 

Student responses to five related questions dealing with linear equations, 
intercepts and slope were examined singly and in combination. On the pre- 
test, 22% of 92 students correctly answered 3 of the 5 questions. Only one 
student correctly answered all five questions. 
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7. The equation of the line with slope -3 and ^intercept 5 is: 
a. y = -5x+3 b. y=5x-3 c.y = - 3* +5 d. y=3x-5 

4. The x-intercept of the equation 2x- 1y = 12 is: 

a. (0, -12/7) b. (-12/7, 0) c. (0, 6) d. (6, 0) 

5. Given the equation y = mx + b, the vertical-intercept is represented by: 
a. y b. m c. x d. b 

3. Given the equation Ax + By = C, the slope of the equation is: 
a. -A/C b. -A/B c. C/A d. C/B 

9. Given the view window and graph shown below, the equation of the line is: 


a. 6x+4y=12 b. 2x-3y=12 c. 6x-4y=12 d. 3x+2y=12 


Table 3. Related pretest responses to questions on linear equations, intercepts and slope. 


QUESTION 

n = 92 

% CORRECT 

7. Given slope -3 and ^intercept 5, select linear 
equation. 

65 

71% 

4. Determine the x-intercept of the equation 
2x- 7y = 12? 

38 

41% 

5. What is the vertical intercept of y = mx + b ? 

37 

40% 

3. What is the slope of Ax + By = C? 

24 

26% 

9. Given the view window and graph, what is the 
equation? 

13 

14% 




3 of 5 questions answered correcdy 

20 

22% 

4 of 5 questions answered correctly 

6 

7% 

All five questions answered correctly 

1 

1% 


pTTTTDUw 
Xnin='3 
Xnax-10 
Xscl=l 
Vnin= '6 
Vnax=4 
Vscl=l 
Xres=l 



Asked to identify the relation between slopes of parallel lines, 72% of 71 
students who took the second pre-test on linear systems gave a correct 
response. Only 41% were able to correctly identify the correct relationship 
between slopes of perpendicular lines and only 22 students (31%) answered 
both questions correctly. 

Sixty-eight percent of the 71 students correctly solved the system: 

3x- y = -2 
3x- y = 6 
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but only 35% of the students were able to determine the point of intersection of: 

y = 2 - x 
3x+ 2y= 2 

Given the graph of two intersecting linear equations, 8 students (11%) 
correctly estimated the point of intersection. 

Three related questions addressed generalisations about linear inequalities. 

6. If y = ax + b, what can be said about the signs of a and of b in the graph 
below? 



a. a > 0 and b > 0 b. a < 0 and b > 0 c. a > 0 and b < 0 d. a < 0 and b < 0 

5. If a and b are any real numbers such that 0 < ab< b, what must be true of ab? 
a. 0 < ab < b b. b < ab < 0 c. 0 < b < ab d. ab < a < 0 

10. What are the values of a and b, where a\b - 2| < 0 ? 

a. a < 0 and b is any number greater than 2. 

b. fir < 0 and b is any number greater than or equal to 2. 

c. a < 0 and b is any number except 2. 

d. a < 0 and b is any number less than 2. 


Table 4. Responses to related pretest questions: Testing generalisations about inequalities. 


QUESTION 

CO 

ii 

£ 

% CORRECT 

6. Given 0 < a < 1 < b, what must be true of ab ? 

47 

64% 

5. What are the signs of a and b, given graph of 
y = ax + b ? 

27 

37% 

10. What are the values of a and b whenever 
a\b- 2| < 0? 

13 

18% 




2 of 3 questions correct 

17 

23% 

All three questions correct 

5 

7% 


(B) Teacher data 

By December 2006 none of the three teachers in the project had incorpo- 
rated curricula changes in their classes as a result of analysing student 
responses to the grouped questions. One of the teachers, Lisa, wrote a note 
to the other two teachers in which she implicitly addresses some of the issues 
raised by Royall, and indicates why addressing those issues can be a problem 
in the course of a semester. Lisa gave her students a quiz on the relationship 
between the coefficients of x and y in an equation for a straight line, and the 
x- and y-intercepts, following class discussion and assignment of an interven- 
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tion task. The excerpts below come from an email Lisa sent to the two other 
teachers in the study, Diane and Nancy, following that quiz: 


I'd like to share some results that I think are pretty exciting. As you know, I 
spent part of my two-hour class last Wednesday trying to address some of our 
mutual concerns about students mistaking the coefficients of x and y in a 
general form equation for the x- and ^intercepts. I’m attaching the handout, 
although I did provide grids — love those graph patches! 

I began with a discussion about the concept of intercepts and how to use the 
“cover-up” method to find them and use them for graphing the lines. We did 
the first one together, them they did the rest of the graphs themselves. Then 
we talked about #5, and concluded that the coefficients were, of course, not 
the intercepts. Last, we “brainstormed” lists for the second page questions. I 
was surprised how many things they wanted to add to each list, even though 
some were duplicates (worded differently). I felt good about the lesson, but I 
decided to check their understanding with a brief quiz on Friday. 

The quiz was about systems as well as the concerns above. I was extremely 
pleased by the results. . . . [All but one] of them got the questions asking for the 
intercepts and slope of a line whose equation was given in general form. 
Finally! I hope it lasts through to the final (and beyond) ! 

We infer from Lisa’s final statement that the quiz results led her to believe 
that most of the students who took the test now understood the relationship 
between horizontal and vertical intercepts and slope, and the coefficients in 
an equation of a straight line. What also seems reasonable to infer from her 
comments is that she did not yet see this data as providing evidence that the 
students had a stable, durable understanding. 

Lisa then addressed a difficulty in utilising classroom data to alter curricu- 
lum in real-time, namely that time constraints can inhibit or prevent a teacher 
from taking action to alter curriculum during semester: 

“My final thought today is that I wish I had the time to analyse and address 
all of the needs of my students in all my classes.” 

Despite her feeling she did not have time to analyse the data in order to 
address student needs in real-time, she felt that the method of analysis was 
helping her with her instruction: 

“I do feel that this project is helping me pinpoint where I should focus my 
instruction.” 

Lisa, like the other two teachers in this project, did not alter her curricu- 
lum during the semester as a result of student answers to questions, but she 
did become mindful of a need to change some aspects of her teaching. The 
student data she gathered led her to believe something about those students, 
and lead her to think about what she should do. The fact that anticipated 
curricula changes are postponed to the next semester does not negate the 
critical formative aspect of the assessment, namely that she had become more 
aware of student difficulties, what the data from students was telling her in 
terms of what she believed about their knowledge, and what she should do as 
a result. 
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Conclusions 


Related grouped questions, dealing with varied aspects of a particular mathe- 
matical concept, can provide classroom teachers with valuable data to help 
them understand the coherent strength or fragility of their students’ knowl- 
edge of that concept. However, it is by no means automatic that a teacher will 
have time to analyse that classroom data in real-time in order to alter curricu- 
lum to better meet student needs. What may have to happen, due to time and 
other constraints, is that a teacher may have to postpone analysing such data 
until the end of a semester, and postpone curriculum change until a follow- 
ing semester. From the perspective we have described in this article this is still 
a powerful form of formative assessment because it promotes curriculum 
change based on mindful reflection on the feedback from carefully struc- 
tured questions. 

When a teacher has time to analyse student data they implicitly or explic- 
itly address Royall’s questions of what they believe now they have this data, 
what they should do now they have this data, and what this data is evidence 
for. We argue that by learning to make these distinctions explicit, a teacher 
can become more mindful and aware of what curriculum changes the data 
suggests, and so engage in productive curriculum change in accordance with 
Cronbach’s understanding of the term “formative assessment”. 

The need to align our perspectives about mathematics and the learning of 
mathematics in order to more effectively communicate with our students is 
essential if we expect to reduce the growing number of students who are not 
successful. These students do not reason as we expect them to, nor do they 
retain knowledge and skills from their previous courses that we assume they 
can demonstrate. Research does not provide us with “a best method of 
instruction,” but it has identified many of the transitional contexts in which 
students’ prior knowledge is no longer adequate and provides us with ques- 
tions that help us better understand how our students are thinking. 
Classroom data can be used to alter what is taught, to whom, and when. Data 
from class tests that examine student answers to questions in pairs, or in other 
joint combinations, as indicated above, can substantially assist a teacher to 
make informed decisions about instructional design. Carefully chosen ques- 
tions, examined this way, can stimulate teachers to be more self-regulatory 
about instructional design, and help transform their thinking on instruction 
that is better aligned to student needs. 
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